Neural Associative Memories

Despite of processing elements which are thousands of times faster than the neurons in the brain, modern computers still cannot match quite a few processing capabilities of the brain, many of which we even consider trivial (such as recognizing faces or voices, or following a conversation). A common principle for those capabilities lies in the use of correlations between patterns in order to identify patterns which are similar. Looking at the brain as an information processing mechanism with { maybe among others { associative processing capabilities together with the converse view of associative memories as certain types of artiicial neural networks initiated a number of interesting results, ranging from theoretical considerations to insights in the functioning of neurons, as well as parallel hardware implementations of neural associative memories. This paper discusses three main aspects of neural associative memories: theoretical investigations, e.g. on the information storage capacity, local learning rules, eeective retrieval strategies, and encoding schemes implementation aspects, in particular for parallel hardware and applications One important outcome of our analytical considerations is that the combination of binary synaptic weights, sparsely encoded memory patterns, and local learning rules | in particular Hebbian learning | leads to favorable representation and access schemes. Based on these considerations, a series of parallel hardware architectures has been developed in the last decade; the current one is the Pan-IV (Parallel Associative Network), which uses the special purpose Bacchus{chips and standard memory for realizing 4096 neurons with 128 MBytes of storage capacity.